Picking a Pooling

We introduce a concept we call “mutual metainformation” as a criterion for determining the maximum statistically significant number of clusters in distributional clustering problems. In distributional clustering, one has data drawn from an ensemble of probability distributions and seeks to group similar distributions together. We focus on the case of multinomials, such as a set of biased coins to be organized into groups with similar bias, given count data from a finite number of tosses of each coin. If one knew in advance the number of groups and which coins belonged to which, one could pool the count data accordingly and perform a Bayesian estimate of each bias using a prior, such as a Dirichlet in the case of a multinomial distribution. When the correct pooling is unknown, the Bayesian approach to this more difficult problem is to extend the method by placing a prior (such as the Chinese Restaurant Process) on the number of clusters and their membership. Our method falls intermediate between a full Bayesian expression on all parameters, and a non-Bayesian approach. It allows a direct expression of the intuition that one merely wants to obtain as many clusters as would turn out to be statistically significant, a posteriori, without further prejudice as to what that number should be. We report numerical experiments with artificial data that support the validity of the approach.